In the field of automobile safety and driving assistance, video systems carried on board vehicles are used for the detection of obstacles—objects or persons—or of events outside this vehicle. Using two onboard cameras, the video system managed by a digital processing system allows the distance between the vehicle and these obstacles to be determined. It is then possible to undertake various functionalities, for example: the detection of obstacles, the detection of hazards, the detection and the recognition of road signs, the continuous white line not to be crossed, or else the detection of cars coming from the other direction. The latter detection may be associated with the management of the lights of the vehicle.
The recognition of these obstacles or events is, furthermore, brought to the attention of the driver by warning of intervention from driving assistance systems. The reliability of the cameras is thus critical and can become decisive, for example when it is required to know in real time whether, in the absence of obstacles detected, the road really is free of obstacles or whether one of the cameras is at least partially obstructed. The detection of the obstruction of the cameras is therefore as important as the determination of good visibility. It should be noted that a frequent cause of obstruction is the condensation of water on the optics of the camera. In general, a detection of obstruction leads to warning the driver of the presence of such condensation and may trigger demisting/defrosting means.
The determination of an amount of obstruction of an onboard camera is dealt with in the patent document US 2013/0070966. In this document, the screen is divided into sectors and the probability of obstruction is analyzed by sector based on a measurement of the number of objects detected by their contour within each sector. This is a method of analysis by image sector.
The detection of the camera obstruction according to this method only offers limited performances: a partial obstruction of the camera is only detected in 75% of cases, the average distance for carrying out this detection being 200 meters. Moreover, at start-up, an average distance of 30 meters is needed to determine the status of the obstruction of the camera.
Using the same approach by sector, the idea of the patent document U.S. Pat. No. 8,116,523 is to generate image data by an “edge map extraction” and a detection of characteristic points based on this data. The characteristic points are categorized according to three detection scan regions respectively arranged at a near distance, a medium distance and far away from the vehicle: a region dedicated to roads, another for the side roads and junctions and a region intended for blind alleys or obstacles. In this way, the number of image processing steps is reduced with respect to the detection of fixed models with a scan of the whole image in order to verify the correspondence of the image to the models.
Other methods have been developed for stereoscopic systems with two cameras allowing additional information on depth of the objects and obstacles of the scene observed by the driver to be provided. The depth of a pixel of an element of this scene is inversely proportional to the shift, otherwise referred to as “disparity”, of the matching pixels from the left and right images corresponding to the initial pixel of the scene and respectively detected by the left and right cameras. A disparity map is composed of the set of disparities between the pixels thus matched.
The generation of the successive disparity maps over time allows the performance of the driving aid applications to be enhanced using depth information for the scene. The use of disparity maps is for example illustrated by the patent documents US 2010/0013908, EP 2 381 416 or FR 2 958 774.
The problem is to correctly match the pixels of the left and right images. Conventionally, the generation of a disparity map is carried out in two steps: the determination of various degrees of matching, also referred to as “matching scores”, for each pair of pixels and the extraction of an estimation of disparity for each pair of pixels.
The first step is carried out by taking into account, for each pixel of a pair being analyzed, pixels within its environment. The scores correspond to the degree of similarity between the pixels of the pair under analysis. The second step allows the most probable disparity, estimated based on the matching scores of this pixel, to be assigned to each pixel from one of the two left or right images, called reference pixel. The set of pixels from the reference image onto which the retained disparities have been transferred constitutes the disparity map of the stereoscopic image.
Generally speaking, three types of method have been developed for producing disparity maps depending on the mode of determination of the scores and the mode of expression of the disparities: local, global and semi-global methods.
Local methods are based on the matching scores of each pair of pixels from each image obtained between the pixels immediately surrounding two pixels to be matched. Various correlation functions may be used (sum of the squared differences, sum of the absolute differences, centered normalized intercorrelation, etc.) in order to then determine the disparities of the matched pixels. For each pair of pixels analyzed, the disparity corresponding to the best score is selected.
These local methods are the simplest and hence occupy fewer resources. They generate high-density disparity maps, in other words with a large fraction of pixels with a disparity considered to be valid, the validity being based on a coherence criterion between the disparities of paired pixels. However, these local methods suffer from a high error rate, notably in the areas of occlusion and in the areas with little texture—for example for a new road.
Global methods consist in optimizing an energy function defined over the whole reference image. The energy function defines the constraints with which the disparity map must comply, for example the continuity of the disparity over the objects. Subsequently, the set of the disparities which minimize this energy function is sought. The Graph-Cut method and Belief Propagation are the most studied global methods.
These methods yield dense disparity images comprising few errors. They are, however, complex to implement and require very significant processing and memory resources which are not very compatible with the onboard hardware constraints.
Semi-global methods are based on the same principle as the global methods but on subsets of the image, namely lines or blocks. The breakdown of the problem of optimization of the energy function into subproblems allows the requirements in processing and memory resources to be decreased with respect to the global methods, but leads to the recurrence of the appearance of artifacts on the disparity map, with a non-negligible error rate and an average—to mediocre-density disparity map (which results from the presence of artifacts).